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Summary 

The  dialectical  inquiry  (DI) ,  the  devil's  advocate  (DA),  and  the  expert 
(E)  approaches  are  three  potential  aids  for  making  decisions  under  uncertainty. 
This  study  examines  the  comparative  effectiveness  of  these  aids.   Furthermore, 
an  objective,  nonemotional  DA  (DA  )  is  differentiated  from  an  emotional, 
"carping-critic"  DA  (DA^) .   Results  indicate  that  when  the  state  of  the  world 
conforms  to  the  assum.ptaons  underlying  a  plan,  the  E  approach  is  found  to  be 
superior  to  the  DI  approach.   When  the  state  of  the  world  :s  opposite  to  the 
assumptions  in  the  plan,  the  DA  and  DI  are  superior  to  the  E  approach.   When 
the  state  of  the  world  is  midway  between  the  assumptions  of  the  plan  and  counter- 
plan,  the  DA^  is  superior  to  both  the  DI  and  the  E  approaches.   The  results 
provide  support  for  the  recommendation  to  use  an  objective,  nonemotional  DA 
approach  in  actual  decision-making  situations. 


Effects  of  the  Expert,  Devil's  Advocate,  and 
Dialectical  Inquiry  Methods  en  Prediction  Performance 

Dialectical  inquiry  (DI)  was  proposed  as  an  aid  to    corporate  decision  iT^aking 
by  Mason  (1969).   Mason,  following  C.  West  Churchman's  (1971)  interpretation  of 
Hegel,  suggested  that  a  dialectical  approach  to  decision  making  should  involve 
examining  a  situation  completely  and  logically  from  two  different  and  opposing 
points  of  view.   First,  a  prevailing  or  recommended  strategic  plan  and  the 
data  which  were  used  to  derive  it  are  identified.   An  attempt  is  made  to 
identify  the  assimiptions underlying  the  plan.   Next,  a  counterplan  is 
identified  which  is  feasible,  politically  viable,  and  generally  credible  but 
which  rests  on  assumptions  which  are  the  opposite  of  those  supporting  the  plan. 
A  structured  debate  is  then  conducted  in  which  those  responsible  for  formulating 
strategy  hear  arguments  in  support  of  both  the  plan  and  the  counterplan.   This 
debate,  in  contrast  to  a  traditional  management  briefing,  consists  of  a  forceful 
presentation  of  two  opposing  plans  which  rest  on  different  interpretations  of 
the  same  organizational  data  bank. 

The  DI  approach  is,  according  to  Mason,  distinctly  superior  to  what  he 
calls  the  expert  (E)  approach  which  is  seen  as  the  most  common  approach  to 
strategic  planning  used  by  business.   In  this  approach  members  of  a  planning 
department  or  consultants  provide  expert  advice  regarding  the  plans  the 
organization  should  follow.   The  planners  make  a  study  of  the  organization's 
enviroTiment  (opportunittes  and  threats)  ,  its  resources  (strengths  and 
weaknesses),  its  personal  values,  and  its  social  responsibilities.   The  result 
of  this  study  is  a  set  of  planning  recommendations  which  are  usually  presented 
to  management  in  the  form  of  a  strategic  briefing  session.   Mason  suggests  that 
the  planning  recommendations  contain  hidden  assumptions  which  are  very 
frequently  not  communicated  to  management.   This  is  one  of  the  most  critical 
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drawbacks  to  this  approach  (Mason,  1969,  pp.  B"06-B407) . 

Another  approach  tc  strategy.'  formulation  is  the  Devil's  Advocate  (DA) 
cechnique.   Mason  (1969)  asserts  that  this  approach  should  be  more  effective 
than  the  E  approach  but  less  effective  than  the  DI  approach.   In  this  approach, 
a  planner  appears  before  the  organization's  management  and  advocates  a  plan 
in  a  manner  similar  to  that  of  the  E  method.   Management  or  another  planner 
then  takes  the  role  of  an  adverse  and  often  carping  critic  of  the  plan. 
An  attempt  is  made  to  determine  all  that  is  wrong  with  the  plan  and  to  expound 
the  reasons  why  the  plan  should  not  be  adopted.   Mason  suggests  that  while  this 
approach  does  expose  some  underlying  assumptions,  it  does  so  in  the  context  of 
what  is  wrong  with  them  rather  than  what  they  should  be.   It  does  not  serve  to 
develop  a  new  managerial  world  view.   For  this  reason,  Mason  feels  that  the  DI 
method  should  be  more  helpful  in  the  strategy  formulation  task  than  the  DA 
approach  (Mason,  1969,  pp.  B407-B408). 

Previous  Research  on  the  Two  Strategy  Formulation  Aids 

Until  very  recently,  previous  research  on  the  benefits  of  DI  had  consisted 
of  three  field  studies.   In  the  first  of  these.  Mason  (1969)  conducted  a  case 

study  on  "RMK  Abrasives."   He  presented  a  plan  and  a  counterplan  before  top 
management.   The  major  hypothesis  of  the  study  was  that  the  management  decision 

makers  would  form  a  new,  encompassing  grasp  of  the  planning  problem  when  exposed 

to  a  DI.   Mason  reports  that  interview  and  questionnaire  responses  from  the  RMK 

managers  indicated  a  favorable  disposition  toward  the  DI  approach. 

Mitroff,  Barabba  and  Kilmann  (1977)  arranged  DI  lectures  for  employees  at 

the  Bureau  of  the  Census  in  Washington,  D.C.   After  the  lecture,  the  Census 

employees  were  segregated  into  five  relatively  homogeneous  groups  wherein 

employees  shared  agreement  on  assumptions  underlying  some  of  the  planning  issues. 


Each  of  the  groups  produced  a  planning  report  and  one  person  from  each  group 
was  selected  to  integrate  the  five  reports  into  a  final  planning  report.   The 
final  version  was  supposed  to  represent  a  synthesis  of  the  various  courses  of 
action  developed  by  each  group.   Mitroff,  et  al.  .  report  the  participants  and 
researchers  identified  several  issues  in  the  integrated  plan  as  exciting  and 
innovative. 

Laurenco  and  Glidewell  (197A)  used  a  DI  to  explain  a  conflict  situation 
between  a  local  television  station  and  its  corporate  headquarters.   The  local 
station  desired  autonomous  local  operations  and  the  headquarters  desired  close 
control.   The  resolution  of  this  conflict  reportedly  led  to  a  synthesis  that 
involved  a  constructive  change  in  the  base  of  power.   Laurenco  and  Glidewell 
(197A,  p.  503)  conclude "The  trend  of  the  synthesis  was  away  from  legitimate 
power  and  its  coercive  supplement  and  toward  identification  and  expertise." 
Thus,  the  three  field  studies  do  provide  some  support  for  the  assertion  that 
the  dialectical  approach  is  perceived  as  effective  by  those  participating  in 
the  planning  process  in  which  it  is  used. 

I'Jhile  field  studies  are  certainly  a  valuable  first  step  in  examining  the 
DI,  it  is  necessary  to  examine  DI  effects  under  more  controlled  conditions. 
Inherent  in  the  field  studies  were  a  large  number  of  uncontrolled  variables 
which  make  it  difficult  to  determine  the  unique  effect  of  one  factor.   In 
addition,  the  DI  was  not  directly  compared  to  alternative  techniques. 

Another  line  of  research  (Cosier  and  Rose,  1977;  Cosier,  1978;  Cosier, 
Ruble  and  Aplin,  1978;  Cosier  and  Aplin,  1979;  Cosier,  1980)  attempted  to 
evaluate  the  value  of  conflict  and  the  comparative  effectiveness  of  the  DA 
and  DI  approaches  in  more  controlled  settings.   With  the  exception  of  Cosier 
and  Aplin  (1979),  these  studies  dealt  with  a  financial  prediction  task  which 
involved  uncertainty  and  "cognitive  conflict"  (interpretive  disagreement). 


Inquir7>'  Methods 


In  these  studies,  subjects  who  were  asked  to  make  Dredictions  of  price 'earnings 
ratios,  were  given  cue  values  of  current  ratios,  inventorv  turnover,  and  debt- 
to-equity  ratios.   These  cue  values  were  developed  ir.  such  a  way  that  there 
was  never  a  perfect  correlation  between  any  of  the  cue  values  or  any  combination 
of  them  and  the  price/earnings  ratio.   In  other  words  there  was  always  some 
uncertainty  built  into  the  prediction  task. 

Using  a  laboratory  paradigm  Cosier  and  Rose  (1977)  examined  the  effects  of 
conflict  on  decision  quality.   Subjects  were  trained  to  make  predictions  of 
price/earnings  ratios  for  a  fictitious  company  using  three  cue  dimensions. 
The  cue  disimensions  consisted  of  three  pieces  of  financial  data.   Cue  #1  was 
the  current  ratio,  Cue  ?!'2  was  the  inventory  turnover,  and  Cue  ^-3  was  the  debt- 
to-equity  ratio  for  the  fictitious  company.   One  group  was  given  feedback 
during  training  to  foster  heaviest  reliance  on  cue  #1,  moderate  reliance  on 
cue  #2,  and  minimal  reliance  on  cue  #3.   The  other  group  of  subjects  was 
trained  in  the  opposite  manner — heaviest  reliance  on  cue  /''3,  moderate  reliance 
on  cue  it2,    and  least  reliance  on  cue  /-l.   After  training,  both  groups  of  subjects 
were  exposed  to  another  decision   maker.   Unknown  to  the  subjects,  this  other 
decision  maker  was  programmed  to  agree  with  the  training  policy  experienced  by 
the  first  group  of  subjects.   Thus,  during  the  prediction  phase  following 
training,  one  group  of  subjects  in  the  low  cognitive  conflict  condition 
interpreted  cue  information  similar  to  a  counterpart,  while  a  second  group  in 
a  high  cognitive  conflict  condition  interpreted  cue  information  highly 
divergent  from  a  counterpart.   During  the  initial  state  of  the  after-training 
prediction  phase  (which  involved  equally  weighted  cues  for  maximum,  accuracy)  , 
subjects  in  the  high  cognitive  conflict  condition  made  significantly  better 
predictions  (less  error)  than  subjects  in  the  low  cognitive  conflict  condition. 


Thus,  Cosier  and  Rose  may  have  evidence  that  the  conflict  generated  bv  -he  D^ 
may  in  fact  De  functional. 

Since  Cosier  and  Rose  (1977)  did  not  directly  compare  the  Dl  tc  alternative 
techniques,  further  research  was  needed.   One  alternative  technique  that  deserved 
■examination  was  the  devil's  advocate  (DA).   Dale  (1975,  pp.  113-llA)  proposes 
"the  staff  man  might  act  as  a  devil's  advocate,  bringing  out  all  the  reasons  why 
the  proposal  should  not  be  accepted.   In  that  way,  the  decision  maker  can  at 
least  be  aware  of  the  possible  perils,  as  well  as  the  advantages  in  any  course 
proposed  to  him."   Cosier  (1978)  conducted  a  study  using  a  controlled  paradigm 
that  allowed  a  comparison  between  the  DA  and  DI .   Subjects  were  presented  with 
planning  information  from  two  "experts."   In  the  DA  condition,  one  expert  recom- 
mended a  plan  of  action  and  another  expert  criticized  that  plan  of  action.   in 
the  DI  condition,  one  expert  recommended  an  action  plan  and  the  other  expert 
recommended  a  conflicting  counterplan.   Subjects  made  predictions  within  three 
distinct  contexts.   Unknown  to  the  subjects,  the  first  context  was  consistent 
with  the  recommendations  in  the  plan,  the  second  context  was  consistent  with  a  com.- 
promise  between  the  plan  and  the  counterplan,  and  the  third  context  was  consistent 
with  the  counterplan.   The  results  indicate  the  DA  method  may  be  preferable  tc 
the  DI.   In  the  third  context,  subjects  in  the  DA  condition  predicted  significantly 
better  than  subjects  in  the  DI  condition.   In  addition,  the  DI  was  not  superior  tc 
the  DA  in  the  other  two  contexts. 

Cosier,  Ruble,  and  Aplin  (1978)  and  Cosier  (.1980)  reported  results  from,  two 
controlled  studies  that  cast  further  doubt  on  the  unqualified  merits  of  the  DI. 
They  found  that  even  though  the  conflict  component  of  the  DI  was  perceived  as  more 
useful  in  some  instances,  in  general  the  DI  did  not  lead  to  significantly  more 
accurate  predictions  when  compared  to  alternative  planning  approaches. 

Finally,  a  field  study  by  Cosier  and  Aplin  (1979)  attempted  to   evaluate  the 
comparative  effectiveness  of  the  DI ,  DA,  and  E  approaches  at  producing 
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planning  statements.   Subjects  were  planners  vithin  the  United  Way  of  America. 
Planners  were  divided  into  h   groups,  each  of  which  was  assigned  to  one  of  four 
treatment  conditions — E  (expert),  DI,  DA,  and  the  C  (control).   A  case  and  a 
"planning  committee  report,"  dealing  with  a  particular  United  V.'ay  Association, 
were  mailed  to  each  of  the  planners  in  the  DI,  DA,  and  E  conditions  giving  a 
description,  brief  analysis  of  the  facts  in  the  case,  and  a  set  of  specific 
recommendations.   In  the  DA  condition,  a  critique  of  the  planning  committee 
report  criticizing  its  underlying  assumptions  but  suggesting  no  alternative 
course  of  action  was  also  given  to  the  planners.   In  the  DI  condition,  rather 
than  a  critique  of  the  planning  committee  report,  the  planners  received  a 
counterplan' based  on  different  assumptions  than  those  underlying  the  report. 
In  the    control   condition,  the  planners  received  no  additional  materials 
beyond  the  case. 

The  planners  were  then  asked  to  prepare  a  planning  document  which  included: 

1)  A  statement  of  the  mission  that  the  planners  felt  was  appropriate  for  this 
particular  United  Way  Association. 

2)  An  analysis  of  the  case  including: 

a)  a  discussion  of  the  problems  and  opportunities  facing  this  Association. 

b)  the  reasons  for  the  planners'  selection  of  these  particular  problems 
and  opportunities. 

3)  A  set  of  specific  recommendations  they  felt  should  be  implemented  in  this 
United  Way  Association. 

The  subjects'  planning  documents  were  evaluated  by  judges.   The  judges  reported 
consistent  low  ratings  for  those  planners  exposed  to  the  DI  condition  across  all 
criteria.   Furthermore,  an  examination  of  the  planning  reports  and  the  judges' 
opinions  revealed  that  the  reports  of  those  exposed  to  the  DI  condition  were 
characterized  by  an  excessive  lack  of  risk-taking  and  an  underutilization  of  United 
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Ways's  resources 

Issues  Examined  in  This  Study 

The  results  of  the  early  field  work  seeir.  tc  be  contradictorv  to  results  of 
the  work  by  Cosier  and  his  colleagues.   There  are  a  number  of  possible  explanations 
for  this  discrepancy,  two  of  which  were  examined  in  the  present  research. 

First,  it  may  be  that  the  way  in  which  the  DA  critique  is  presented  is 
different  in  the  lab  studies  than  in  actual  organizations.   In  organizations,  the 
DA  may  frequently  involve  a  very  strong  criticism  of  the  accuracy  of  the  E's 
interpretation.   In  contrast,  the  DA  critique  in  the  controlled  studies  has  been  a 
rather  mild  statement  that  the  E's  position  should  be  questioned.   In  the  present 
research,  two  separate  DA  statements  were  given  to  the  subjects:  one  was  a  rather 
mild,  "objective"  DA  statement  and  the  other  was  a  strong,  "emotional"  DA  statement. 

Second,  the  use  of  the  DI  in  the  early  field  work  was  accompanied  by  strong 
attempts  to  persuade  those  using  the  technique  of  its  value.   It  may  be  that 
persuasion  has  a  differential  effect  on  the  effectiveness  of  the  various  techniques. 
In  the  present  research,  the  effects  of  a  persuasive  statement  about  the  value  of  the 
techniques  was  assessed  by  giving  half  of  the  subjects  using  each  technique  a 
persuasive  statement  regarding  the  value  of  the  technique. 

This  study  also  attempted  to  replicate  the  principal  findings  in  the  Cosier 
(1978)  study.   The  effectiveness  of  the  objective  DA  versus  the  DI  in  a  context 
favorable  to  the  counterplan  was  examined.   As  we  noted  earlier.  Cosier  (1978) 
found  that  the  DA  was  associated  with  better  decisions  than  the  DI  in  this  context. 
Also,  the  effectiveness  of  the  E  (an  expert  recommending  a  plan)  versus  alternative 
inquiry  methods  was  examined.   Surprisingly,  Cosier  (1978)  found  no  significant 
differences  between  the  E  and  the  DA  and  DI  techniques  in  a  context  favorable  to  the 
plan.   This  lack  of  E  superiority  is  important  when  coupled  with  evidence  that  the 
DA  may  be  superior  when  the  counterplan  represents  reality.   Cosier  (1978)  noted 
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that    in  his    study    the   DA  approach  was   never   dysfunctional    and   seemed    tc  be 
helpful   in   the    counterplan-best    context.      Thus,    the   DA  was    recommended   over   the 
E  without    qualification   of    context. 

Application   of    the   Multiple-Cue   Probability   Learning   Paradigm 

The  Multiple-Cue   Probability      Learning  Paradigm    (MCPLP)   was    employed   to 
examine    the    relative   effectiveness    of    the    inquiry  methods.      In   order   to   attempt 
replication   of   Cosier's    (1978)    findings,    the   MCPLP  was   necessary.       In   addition, 
the   MCPLP,    which  has   been    used   to   address    a   variety    of    interesting   Social 
Judgment   Theors-    concerns    (Brehmer,    1976    and    1979;    Hammond,    1973;    Hammond,    Stewart, 
Brehmer,    and   Steinmann,    1975)    has    some    distinct    benefits.      First,    because    the 
MCPLP   is   based   in   the   Brunswick   Lens  Model    (Brunswik,    1955),    the   decision  maker  is 
faced  with   inferring   the   values    of  a   "hidden"    criterion  variable  based   on    cue 
information   from   the  environment    (Mitroff,    19  74).      In   addition,    the  MCPLP   allows 
the   environment   to   be   programmed  with   uncertainty.      Inferring   the   nature   of  a 
future    "state    of   the   world"   within   an    uncertain   environment    is    a    central   element 
of   strategic   decision  making.      Mason    (1969,    p.    B403)    specifically   notes    that    the 
strategic    decision  maker   "is    concerned  with    future    states    of    the  world   and  hence 
makes    predictions    about    them. " 

Second,    the  MCPLP   allows    objective   measures    of   performance    and   experimental 
control  that  may   reveal   differences   in   the   inquiry  methods-task   relationships. 
The   inquiry  methods   are   intended   as    techniques    for   influencing   the    cue-utilization 
policies    of   decision  makers.       If    a   particular    inquiry   method   is    superior   in 
assisting   decision  makers    to   "read"    their   environments,    then   this    should   be 
reflected  by   a  high   degree    of   prediction  accuracy. 1       As   suggested   by   Brehmer 
(1976),      the   opportunity    to   examine    the    task-policy    linkage    under    controlled 
conditions    is    inherent    in   the   MCPLP.       In    fact,    prediction   accuracv    reflects    the 
difference  between    the    decision   maker's    oolicv   for   using   cue    information    and   the 
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actual  cue-criterion  relationships  defined  by  the  task. 

There  are  at  least  two  limitations  imposed  by  this  application  of  the 
MCPLP.   In  this  study,  the  MCPLP  is  onlv  relevant  for  situations  that  involve 
prediction  decisions  over  some  period  of  time.    Hence,  it  cannot  be  applied  tc 
"one-shot"  decisions.   Also,  an  environment  that  is  random  will  not  allow 
specification  of  cue-criterion  probability  relationships.   Thus,  decisions 
under  total  uncertainty  are  not  addressed  by  this  application  of  the  MCPLP. 

METHOD 

Ninety-six  subjects  were    "run"    during   a   one-week  period.       Subjects,    v.'ho 

were  undergraduates  from  two  organizational  behavior  classes,  participated  in 

the  experiment  in  order  to  satisfy  a  course  requirement.   The  subjects  faced  a 

series  of  financial  prediction  situations.   Each  situation  was  defined  by  three 

cue   values:      the   current    ratio    (x   ) ;    inventory    turnover    (x^);    and   debt-to-equity 

ratio    (x   ) .      Each   prediction   situation   required   that    the   subject   predict    a   price/ 

earnings    (P/E)    ratio    (v    ) .      As   noted  bv   Cosier   and   Rose    (1977)    these    financial 

'  s 

labels  were  chosen  because  there  is  no  empirical  evidence  to  suggest  any  different 
_a  priori  importance  of  the  three  cues  and  the  labels  should  induce  high  subject 
involvement  in  the  task.   In  addition,  subjects  were  told  in  the  written  instructions 
to  learn  how  to  use  the  data  as  presented  in  this  study  to  make  their  predictions. 

All  subjects  received  a  packet  of  materials.   Each  packet  contained  written 
instructions,  planning  information,  cue  cards,  and  a  response  sheet.   All  instruction 
and   plans   were  presented  in  written  fonri.   Subjects  were  told  they  would  be 
predicting  P/E  ratios  for  three  independent  profit  centers  within  ABC  Electronics. 
Each  profit  center  involved  20  monthly  predictions  and  each  center  was  to  be 
treated  as  an  independent  operation.   Subjects  were  told  they  had  three  sources 
of  information  to  aid  their  predictions.   Source  one  was  the  financial  cue  values, 
source  two  was  feedback  consisting  of  the  correct  P/E  value  after  each  monthly 
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prediction,  and  source  three  was  planning  information  provided  by  staff  experts. 

Each  profit  center  consisted  of  a  distinct  group  of  20  index  cards  'or  cue 

cards).   Side  one  contained  the  three  cue  values  (current  ratio,  inventory  turn- 

overn  and  debt-to-equitv  ratio)  and  side  two  contained  the  correct  ?/E  ratio  (v  ")  . 

■         ■  c' 

The  cue  and  criterion  values  were  whole  numbers  which  ranged  from  1  through  20. 
Subjects  were  given  a  response  form  that  contained  three  columns  of  20  spaces  for 
making  monthly  predictions.   The  procedure  was  explained  as: 

Starting  with  the  first  card  in  your  deck,  each  monthly  prediction 

is  done  as  follows: 

1.  Recall  the  planning  information; 

2.  Consider  the  three  "cue"  values  on  side  1  of  the  card; 

3.  Recall  any  relevant  historical  information  (after  the  first 
prediction  for  each  profit  center  you  will  have  historical 
information  or  feedback) ; 

4.  Record  your  prediction; 

5.  Turn  the  card  over  and  view  the  correct  response; 

6.  Continue  to  the  next  card. 
Inquiry  Methods 

Each  subject  was  randomly  assigned  to  one  of  the  four  inquiry  method  condi- 
tions.  In  the  E  condition  subjects  were  told  they  would  consider  information  from. 
a  planning  report  written  by  "Expert  A."  The  plan  told  each  subject  that  based 
on  Expert  A's  research  primary  weight  should  be  placed  on  the  current  ratio  when 
predicting  P/E  ratios.  Subjects  were  told  this  opinion  was  based  on  "analysis  of 
past  data  and  the  fact  that  the  current  ratio  incorporates  information  about  cur- 
rent assets  and  current  liabilities."   Expert  A  further  recommended  that  a  moder- 
ate amount  of  weight  be  placed  on  the  inventory  turnover  value  and  little  or  no 
weight  be  placed  on  the  debt-to-equity  ratio. 
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In  the  DI  condition  subjects  were  tcid  they  would  consider  infornation  fror. 
a  plan  and  a  conflicting  counterplan.   The  counterplan  was  preuared  bv  Ey.uert  B 
and  was  in  fact  a  contradiction  of  the  plan  prepared  by  Expert  A  (the 
plan  was  the  same  document  in  all  conditions) .   The  counterplan  did  not  directlv 
criticize  the  plan,  but  instead  was  based  upon  conflicting  assumptions.   For 
example,  Expert  B  stated  that  his  analysis  of  past  data  dictated  that  primary 
weight  should  be  placed  on  the  debt-to-equity  ratio  when  predicting  P/E  ratios. 
Expert  B  recommended  moderate  weight  be  placed  on  the  inventory  turnover  value 
and  little  or  no  weight  be  placed  on  the  current  ratio.   As  was  the  case  for 
Expert  A,  Expert  B's  rationale  was  based  on  a  description  of  the  three  cues  and 
prior  research. 

In  the  DA   condition,  subjects  were  given  a  plan  written  by  Expert  A  and  a 
critique  of  the  plan  written  by  "Expert  B."   The  critique  questioned  the  conclu- 
sions and  assumptions  of  Expert  A  but  did  not  offer  alternative  propositions.   For 
example,  regarding  the  importance  of  the  current  ratio,  Expert  B  noted  his  analysis 
revealed  it  was  questionable  to  put  primary  weight  on  the  current  ratio  and  "It  is 
quite  possible  that  a  high  correlation  may  not  exist  between  current  ratios  and 

P/E-ratios."   All  of  the  experts  in  this  study  were  defined  in  the  instructions 
as  staff  assistants  employed  by  ABC  Electronics  to  assist  in  planning. 

Finally,  in  the  DA„  condition,  the  subjects  were  given  a  plan  and  critique 
as  in  the  DA.,  condition.   However,  in  this  condition,  the  critique  was  worded  much 
more  strongly  and  was  much  more  critical  of  Expert  A  than  was  the  critique  in  the 
DA-  condition.   For  example,  regarding  the  importance  of  the  current  ratio,  Expert 
B  stated,  "Expert  A's  assumption  that  there  should  be  a  high  correlation  between 
known  current  ratios  and  future  P'E  ratios  is  certaintly  not  true  in  all  cases 
and  represents  a  dangerous  overgeneralization  which  could  lead  tc  errors  in  pre- 
diction." 
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An  attempt  was  made  during  all  manipulations  to  capture  the  fundamental  pro- 
cess defining  each  inquiry  method.   For  example,  the  conflict  inherent  in  com.peting 
recommendations  was  deemed  the  fundamental  Dl  component  (Mason.  1969).   The  dis- 
sent evidenced  by  a  critical  opinion  was  isolated  as  the  key  DA  component  (Mason, 
1969). 

Persuasion 

In  addition  to  being  assigned  to  one  of  the  four  inquiry  method  conditions 
each  subject  was  randomly  assigned  to  one  of  two  persuasion  conditions.   Subjects 
in  the  weak  persuasion  condition  were  simply  given  a  statement  describing  the  in- 
quiry method.   There  was  no  attempt  to  persuade  them  of  the  value  of  the  method. 
Subjects  in  the  strong  persuasion  condition  were  given  a  description  of  the  in- 
quiry method  they  were  to  use  as  well  as  a  summary  of  the  past  uses  of  the  tech- 
nique, and  its  proven  value.   Finally,  subjects  in  mis  "persuasion"  condition 
were  asked  to  make  strong  effort  to  use  the  technique  to  improve  their  predictions. 

Context  of  the  Decisions 

Unknown  to  the  subjects,  the  three  independent  profit  centers  represented 

three  different  decision-making  contexts  or  "states  of  the  world."   State  one  (SI) 

was  programmed  such  that:   r     =  .80,  r     =  .50,and  r     -  . ^0   This  srate 

l^c         ''2^c  Vc 

was  constructed  to  be  in  general  agreement  with  the  plan  offered  by  Expert  A. 

Expert  A's  plan  is  in  fact  predicated  on  assumptions  reflected  in  the  .8-. 5-. 2 

correlation  scheme.   Notice  world  state  schemes  are  "hidden"- from  direct  view  of 

the  subjects.   S3  was  created  so  that  r     =  .20,  r     =  .  50  and  r     :=   "0 

Thus  S3  was  in  essential  agreement  with  the  assumptions  and  recommendations  in  the 

counterplan.   A  "compromise"  state  was  represented  by  S2.   The  cue  weights  in  ?2 


were  created  so  that  all  three  cues  would  be  of  equal  importance:   r 


^^1^'c 


^x  V     ~  •^^>  3'^<^  ^x  V   ~  -55.   Since  mult icollinearity  was  near  zero,  the  coeffi- 


-2'c  Vr 
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2 

cient   of   determination    (R^      )  in   all  wcrld   states  v.-'as   r^rin-arilv   deterr.ined  b'-   the 

XV  •  •  ■ 

•  c  ^  3  , 

three  cue-criterion  correlation  schemes  in  each  world  state  (R~   -   I    r~ 

x>-  c   ,•  _  -   ^  •  y  „ 

2  "   1--    1-  c 

In  all  three  world  states  R_   -  .88.   Notice  unexplained  error  variance  (ranccr-.- 

'"  c 
ness)  was  present  in  all  states  of  the  world. 

In  order  to  prevent  confounding  world  states  with  sequence,  the  states  i:cre 

assigned  from  a  latin  squares  scheme.   This  created  a  between-subj ects  factor  that 

will  be  called  "order,"   Order  consists  of  three  conditions:   SI,  S2,  S5;  S3,  SI, 

b^^    b2,    bj,    J  A.  , 

Dependent  Measure 

The  dependent  measure  used  to  assess  prediction  accuracy  is  the  average  abso- 
lute  error  for  each  world  state  (i,e,  — '  ■  ^-        where  v   is  the  subject's  predic- 

20  "  S  J  r 

tion  and  y   represents  the  criterion  value  per  month) ,   This  measure  was  used  bv 
^  c 

Cosier  and  Rose  (1977),  and  Cosier (1978  and  1980) as  an  indicator  of  prediction  oer- 
formance.   In  order  to  avoid  some  difficulties  associated  with  absolute  measures 
all  subjects  considered  the  same  cue  values  in  each  state  of  the  world.   In 
addition,  all  subjects  were  told  this  measure  was  t'ne    criterion  for  performance. 
Although  this  dependent  variable  has  the  advanta^^e  of  being  an  "objective" 
performance  measure,  some  information  is  admittedly  "lost"  within  each  block  of 
20  predictions.   Unfortunately,  we  cannot  break  the  20  trials  into  smaller  blocks 
and  maintain  equal  and  comparable  sets  of  cue-criterion  relationships.   The 
"correct"  answers,  which  invqlve  some  error  due  to  uncertainty,  are  based  on  a  20 
trial  constraint.   The  total  number  of  predictions  in  the  study  (60),  however, 
were  broken  into  3  sets  of  20  to  allow  some  insi£ht  into  performance  effects  over 
time. 
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RESULTS 
Manipulation  Checks 

In  order  to  determine  if  the  subjects  seemed  to  read  the  planning  informa- 
tion, an  item  on  the  post-task  questionnaire  asked  which  cue  dimension  that  Expert 
A  recommended  be  highly  emphasized.   Four  specific  possibilities  (current  ratio, 
inventory  turnover,  debt-to-equity  ratio,  and  P/E  ratio!)  and  an  "I  don't  know" 
choice  were  available.   Ninety-nine  percent  of  the  subjects  provided  the  correct 
response  (the  current  ratio).   The  subjects  were  also  asked  if  they  treated  each 
profit  center  as  independent  of  the  others.   Seventy-five  percent  agreed  or  strongly 
agreed  with  the  statement  that  the  profit  centers  were  independent  (S%   were  neutral) 
Thus  the  vast  majority  of  subjects  were  able  to  recall  E>rpert  A's  primary  recom- 
mendations and  reported  treating  the  profit  centers  as  independent  units. 

An  item  on  the  post-task  questionnaire  asked  whether  the  subjects  perceived 
a  strong  attempt  at  persuasion.   A  continuous  10  cm.,  line  was  anchored  by  "Yes, 
a  strong  attempt"  and  "No,  no  attempt."   Subjects  in  the  persuasion  condition  re- 
ported a  significantly  stronger  attempt  (F  =  7.73,  p  <    .01)  than  subjects  in  the 
no-persuasion  condition. 

Finally,  subjects  were  asked  to  circle  the  statement  that  described  the  in- 
formation that  they  received  from  the  expert (s).   Five  choices  were  available — 
1)  Two  divergent  and  conflicting  sets  of  recommendations;   2)  Two  different  but 
non-conflicting  sets  of  recommendations;   3)  One  set  of  recommendations  and  a 
critique;   A)  One  set  of  recommendations  and  a  supportive  statement;  and  5)   A 
single  set  of  recommendations.   Seventy-nine  percent  of  the  DI  subjects  picked  1), 
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and  88?  of  the  E  subjects  picked  'j)  or  5*.   Interestingly,  v.'hile  bl','-.   of  the  DA 
subjects  selected  2)  or  3),  onlv  37%  of  the  DA^  subiects  selected  2)  or  3). 
Sixty-three  percent  of  the  DA^  subjects  selected  1)  as  best  describing  their 

information.   These  data  show  that  the  perceptions  of  the  experts'  information 

2 
did  vary,  depending  upon  the  inquiry  method  CX.  =  93.0,  p  <  .001).   Furthermore, 

the  most  frequent  reports  of  "conflicting  recommendations"  were  present  in  the  DI 

and  DA„  conditions. 

Prediction  Performance 

A  four-factor  ANOVA  consisting  of  three  between-sub jects  factors  (inquiry 
method,  persuasion,  and  order)  and  one  within-subjects  factor  (context)  was  used 
to  analyze  the  performance  data  for  the  subjects. 

Table  1  shows  that  a  context  X  inquiry  method  interaction  (F  =  5.39,  p  <  .01) 
as  well  as  a  context  X  order  interaction  (F  =  5.06,  p  <  . 01)  were  present .   These 
interactions  required  simple-effects  analyses  for  interpretation. 


Insert  Table  1  Here 


Regarding  the  inquiry  method  X  context  interaction,  significant  inquiry 
method  effects  were  found  for  each  context.   The  Duncan  multiple  range  technique 
revealed  that  within  the  si  context  (plan  best)  the  subjects  exposed  to  the  E 
inquiry  method  predicted  significantly  better  (p  <  .05)  than  those  exposed  to  the 
DI  (see  Table  2  for  the  mean  absolute  error  values).   Within  the  S^  context  (coun- 
terplan  best)  the  subjects  exposed  to  the  DA  and  DI  predicted  significantly  bette: 
(p  <  .01)  than  subjects  exposed  to  the  E  inquiry  method.   Within  the  S(^  context 
(compromise  between  the  plan  and  counterplan)  the  superiority  of  the  subiects 
exposed  to  the  DA  over  the  DI  and  the  E  inquiry  methods  was  significant  at  the 
.05  level. 


Insert  Table  2  Here 

As  noted  earlier,  a  context  X  order  interaction  was  also  found.   Significant 
simple  effects  were  found  for  contexts  at  each  order.   Using  the  Duncan  multi- 
ple range  technique,  subjects  using  Order  1  (SI.  S2.  S3)  predicted  significantly 
better  in  S3  than  either  SI  or  S2  (p  <  .01).   In  Order  2,  (S3,  SI,  S2)  Drediction 
error  was  significantly  less  in  S2  than  in  S3  (p  <  .01)  and  less  in  SI  than  S3 
(p  <  .01).   In  Order  3,  (S2,  S3,  SI)  prediction  error  was  significantly  less  in 
SI  than  in  S2  or  S3  (p  <  .01  and  .05  respectively).  In  addition,  subjects  pre- 
dicted significantly  better  in  S3  than  S2  (p  <  .01). 

These  data  suggest  subjects  improved  their  predictions  over  time.   In  order 
to  verify  this  possibility,  a  supplementary  analysis  was  run  examining  "sequence" 
(position  of  context  over  time)  as  a  within-subjects  factor.   This  analysis  re- 
vealed a  sequence  main  effect   (F  =  9.97,  p  <  .01).   This  effect  reflected  predic- 
tion improvements  (at  .01)  between  each  set  of  20  predictions  over  time. '- 

DISCUSSION 

The  conclusions  from  the  present  study  are  basically  consistent  with  those  of 
the  earlier  work  by  Cosier  and  his  colleagues.   In  this  study,  there  was  a  signi- 
ficant context  X  order  interaction  as  well  as  a  significant  context  X  inquiry 
method  interaction.   The  context  X  order  interaction  and  the  subsequent  examination 
of  "sequence"  demonstrates  that  the  subjects  were  able  to  improve  their  prediction 
performances  over  time. 

The  context  X  inquiry  method  interaction,  and  subsequent  analysis,  suggest 
that  in  situations  in  which" an  expert's  plan  reflects  the  true  state  of  the  world, 
the  DI  technique  is  actually  detrimental  to  prediction  performance.   In  this  case, 
simply  giving  individuals  the  expert's  analysis  and  plan  is  likely  to  lead  to 
more  accurate  prediction  performances  than  giving  individuals  DI  information. 
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Interestingly,  this  superiority  of  the  E  approach  ever  the  DI  ir.  SI  was  not 

found  by  Cosier  (1978). 

It  is  important  to  note,  however  that  the  E  method  was  not  significantly  superior 

to  either  DA  technique   in  SI.   This  finding,  which  is  consistent  with  results  in 

Cosier  (1978),  suggests  that  the  DA  approach  may  not  impair  decision  caking  under 

the  plan-best  condition. 

When  the  expert's  plan  represents  the  opposite  cf  the  true  state  of  affairs. 

both  the  "objective"  DA  (DA  )  and  the  DI  methods  of  presenting  planning  inforrr.a- 

tion  may  be  superior  to  the  E  approach.   Thus,  the  conflict  introduced  by  both 

methods  is  apparently  helpful  in  situations  where  the  true  state  of  the  world  is 

quite  different  from  the  assumptions  with  the  expert's  plan. 

Finally,  when  the  true  state  of  the  world  is  midway  between  the  assumptions 

contained  in  the  expert's  plan  and  those  contained  in  the  counterplan,  the  objec- 
tive DA  method  (DA.)  may  be  superior  to  both  the  E  and  the  DI  approaches; 

Mason  (1969)  has  criticized  the  DA  approach  on  the  grounds  that  it  involves 
simply  the  criticism  of  a  plan  with  no  suggestions  of  alternative  plans.   It  may 
be  that  this  feature  of  the  DA  approach  is  actually  a  strength  rather  than  a 
weakness.   Planners  faced  with  two  contradictory  sets  of  analysis  and  recommenda- 
tions may  in  fact  be  tempted  to  make  a  choice  between  them,  either  fully  accepting 
the  assumptions  underlying  the  plan  or  the  counterplan.   When  the  plan  represents 
the  true  state  of  the  world  this  would  explan  why  the  E  approach  is  superior  to  the 
DI.   The  DI  approach  may  lead  some  subjects  to  initially  choose  the  counterplan 
which  would  lead  to  poorer  prediction  performance.   KTien  the  plan  represents  the 
opposite  of  the  true  state  of  the  world,  however,  the  subjects'  choice  of  the 
counterplan  would  lead  to  superior  prediction  performance,  explaining  why  the  DI 
approach  may  be  superior  to  the  E  approach  in  this  situation. 


Inquiry-  Ilethods 
19 

The  DA  approach,  on  the  other  hand,  does  not  encourage  a  planner  tc  make  a 
choice  between  two  contradictory  plans.   Rather,  it  encourages  the  planner  to 
examine  the  assumptions  underlying  the  expert's  plan.   Subjects  using  this  approach 
might  have  a  greater  tendency  to  develop  assumptions  based  on  data  fron  the  environ- 
ment rather  than  accepting  the  assumptions  of  either  a  plan  or  a  counterplan.   This 
would  help  to  explain  why,  in  a  situation  in  which  the  state  of  the  world  conforms 
to  neither  the  plan  nor  the  counterplan,  the  DA  technique  was  found  to  be  superior 
to  both  the  E  and  the  DI  approaches. 

These  data  provide  some  support  for  a  point  made  by  Mason  (1959).   He  sug- 
gested "that  a  potential  problem  with  the  DA  is  that  it  may  be  perceived  by  decision 
makers  as  an  overly  negative,  carping  kind  of  criticism  and  that  this  perception 
may  reduce  the  effectiveness  of  the  DA.   In  this  study,  the  DA  •  critique  was  worded 

as  a  reasonably  objective,  unemotional  statement  while  the  DA^  did  have  elements 

of  carping  criticism.   The  DA^ ' s  failure  to  be  associated  with  superior  performances 

when  compared  to  the  DI  and  E  indicates  that  this  strong  emotional  condition  may 
be  detrimental  to  the  effectiveness  of  the  DA  in  some  circumstances.   This  leads 
to  the  suggestion  that  when  the  DA  is  used,  care  should  be  taken  to  avoid  the  nega- 
tive, carping  sort  of  criticism  Mason  (1969)  condemns. 

This  study  provides  evidence  that  the  relationship  between  the  task  and  the 
inquiry  method  is  crucial.   When  comparing  the  E  and  the  DI,  this  study  suggests  the 
"best"  method  is  unclear.   In  SI,  the  task  is  best-matched  with  the  E  technique. 
In  S3,  however,  the  task  is  best-matched  with  the  DI.   Since  it  is  difficult  to 
specify  the  context  before  the  decision  is  made,  the  choice  between  E  and  DI  is 
problematic.   The  "objective"  DA,  however,  has  not  been  significantly  inferior 
to  the  E  in  SI.   In  addition,  it  has  been  significantly  more  helpful  than  the  E 
and  DI  techniques  in  other  contexts.   Thus,  over  all  contexts,  the  DA  approach 
appears  to  be  at  least  as  good,  or  better  than  the  alternative  inquiry  methods. 
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It  can  also  be  argued  that  due  to  the  uncertainly  generally  associated  vith 
organizational  decision  making,  it  is  reasonable  to  suspect  that  the  assumptions 
underlying  the  plan  will  not  all  be  correct.   Nor  is  it  likely  that  all  of  the 
assumptions  underlying  the  counterplan  will  be  correct.   In  many  decision 
situations,  the  true  state  of  the  world  may  lie  somewhere  between  the  plan  and 
the  counterplan, the  situation  in  which  the  objective  DA  was  shown  to  be  superior 
in  the  study. 

Interestingly,  the  persuasion  factor  had  little  effect  on  prediction  perfor- 
mance.  Telling  subjects  that  a  technique  was  useful  did  not  result  in  increased 
objective  performance.   Criticisms  of  previous  controlled  studies  suggesting  the 
results  are  of  limited  generalizability  because  of  a  lack  of  an  indoctrination 
program  may  be  unwarranted. 

Of  course,  one  can  argue  that  the  operationalizations  of  the  factors  in  this 
study  do  not  accurately  represent  the  "real-world"  inquiry  methods.   This  possi- 
bility, which  is  characteristic  of  any  controlled  study,  suggests  the  need  for 
future  research  using  a  variety  of  paradigms  before  the  DI  is  offered  to  prac- 
titioners as  an  aid  for  making  corporate  decisions.   Case  studies  and  field 
experiments  may  prove  helpful.   These  methods,  however,  may  lack  the  control 
over  confounding  factors  and  clear  performance  measures  available  from  laboratory 
designs.   Continuing  research  into  the  DI  may  help  us  decide  whether  models, 
such  as  "Strategic  Assumptions  Analysis"  (Mitroff  and  Emshoff,  1979)  that  assume 
the  validity  of  the  DI,  are  premature.   It  may  be  that  an  objective,  factual- 
based  DA  component  should  be  offered  in  lieu  of  a  DI  component  in  strategv  models. 
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FOOTNOTES 


Inferences  about  inquiry  method  differences,  of  course,  require  randoni 
assignment  and  common  task  situations  across  subjects. 

Sequence  is  merely  a  reordering  of  the  contexts  (when  required)  in  temporal 
order.   Sequence  was  not  run  at  the  same  time  V7ith  the  context  factor  because  of 
the  lack  of  independence  between  the  two  factors.   A  sequence  X  context  interaction, 
therefore,  makes  little  sense  given  the  nature  of  the  factors. 


Table  1 
ANOVA  for  Prediction  Performance 
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Source 


df 


MS 


Subjects 

Inquiry  Method  (B) 

Persuasion  (C) 

Order 

(D) 

B  X  C 

B  X  D 

C  X  D 

B  X  C 

X  D 

Error 

(Between) 

Within 

Context  (A) 

A  X  B 

A  x  C 

A  X  D 

A  X  B 

X  C 

A  X  B 

X  D 

A  X  C 

X  D 

A  X  B 

X  C  X  D 

Interaction 

Total 

95 

3 

1 

2 

3 

6 

2 

6 

72 

192 

2 

6 

2 

A 

6 

12 

4 

12 

144 

287 


1.05 

2.10 

0.20 

2.95 

0.27 

1.76 

.97 

.60 

.98 

.37 

.16 

1.65 

.15 

1.55 

.32 

.  .43 

.22 

.20 

.31 

.60 


2.18 
.20 

3.00 
.27 

1.79 
.99 
.61 


.51 
5.39* 

.50 
5.06* 
1.05 
1.42 

.73 
.66 


.01 


Table  2 

Inquiry  Method  and  Order  Mean  Absolute 
Error  Values  at  Each  Context 
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Inquiry  Method 


Order 


Context 

DI 

°*i 

DA^ 

E 

J- 

-,         < 

^ 

J 

SI 

2.98 

2.56 

2.72 

2.41 

2.70 

2.58 

2.72 

S2 

2.91 

2.38 

2.76 

2.91 

2.65 

2.46 

3.12 

S3 

2.54 

2.45 

2.77 

3.16 

1 

2.43 

2.86       i 

1 

2.90 

